The invention relates to the field of handling data packets that are transmitted over a network. Especially the invention concerns the subject of intercepting data packets, i.e. providing access to essentially all data packets sent and received by a certain system.
The growth of the Internet and its use for new applications have made it beneficial to introduce new services that affect the way network packets are transmitted through a network. Examples of products and services with such needs include:
Network-level encryption applications, such as VPNs (Virtual Private Networks), as described for example in the reference marked as SWE98 in the enclosed list of references. All of the enclosed list of references is hereby incorporated by reference. These applications encrypt and decrypt data packets as they are transmitted in and out of a system to provide security for the data in transmission. VPNs are essential for reliably conducting commercial activity (buying and selling) through a publicly available packet-switched data transmission network such as the Internet, and for using Internet for mission-critical business applications.
Firewalls, as described for example in the reference marked as CB94 in the enclosed list of references. These are network security devices that filter network traffic according to specified criteria, allowing only some packets to pass through. Firewalls are usually implemented as extensions to general-purpose operating systems, so that they can monitor and alter traffic flowing through the system, but may also be implemented as dedicated hardware devices.
Intrusion detection and packet sniffing. Many intrusion detection and network monitoring tools need access to the data transmitted in packets through a network. Similar tools are also used to collect statistics about network traffic (e.g. as described in the reference marked as Waldbusser97 in the enclosed list of references).
Multimedia applications. It is predicted that 25% of the global telecommunications market value will be IP-based (where IP comes from Internet Protocol) in just a couple of years, and much of the required data traffic will be transmitted over the public Internet using general-purpose computers as terminals. Guaranteed QoS (Quality of Service) is essential for interactive video and audio applications over such networks, as described for example in the reference marked as SCFJ96 in the enclosed list of references.
Mobility of terminals. Mobility is becoming increasingly important also for packet-switched data transmission networks and the Internet, as described for example in the reference marked as Perkins96 in the enclosed list of references. In many cases, mobility support will be provided as added value to an existing system, and it will require the capability of modifying and redirecting incoming and outgoing data packets.
All of the above mentioned applications use specific protocols that are not available in all widely used operating systems. In many cases, vendors will want to provide support for these services on widely installed platforms for which no support for them is readily available. Implementing such support often requires that the implementor gets access to all data packets sent and received by the system. The module that provides such access is called a packet interceptor. Such modules typically also provide some information about the available network interfaces and their configuration (e.g. network addresses) to the application. The application in turn usually consists of a kernel-mode component that handles real-time packet processing, and a normal user-mode application for management and other functions that are not time-critical and/or require user interaction.
Overall, the need for intercepting packets flowing in and out of a system is becoming extremely important. This has been recognized by programmers and commercial operators in the field, as well as operating system vendors such as Microsoft Corporation. A substantial amount of work is made to implement packet interception functionalities in networking systems and related products.
Existing solutions for the packet interception problem fall mostly into the following categories A), B) and C):
A) Intermediate drivers. A TCP/IP (or other) protocol stack, where TCP/IP comes from Transmission Control Protocol/Internet Protocol, is usually layered so that network device drivers provide a standard interface to a particular hardware device known as a network adapter, and protocol stacks implement various network protocols. The protocol stacks are made hardware-independent by the standard interface, which the device drivers must implement. In Windows operating systems, where Windows is a registered trademark of Microsoft Corporation, this interface is called NDIS (Network Driver Interface Specification), as described for example in the reference marked as Win4DDK in the enclosed list of references. In Sun Solaris, which refers to the registered trademarks Sun, Solaris and Sun Solaris of Sun Microsystems, similar functionality is provided by the STREAMS interface, as described for example in the reference marked as STREAMS93 in the enclosed list of references.
Intermediate drivers are readily supported under at least Microsoft Windows NT 4.0 (registered trademark of Microsoft) and Sun Solaris operating systems. Microsoft has even provided sample code for developing intermediate drivers for applications such as those described above. At least two such samples are available, and many vendors have implemented products based on this technology.
B) WINSOCK interception. It is well known in the industry that several products replace the WINSOCK.DLL, as described for example in the references marked as Bonner96 and QS96 in the enclosed list of references. WINSOCK.DLL is a file on Windows systems. Some products use intermediate drivers at the LSP (Layered Service Providers) level, as described for example in the reference marked as Win4DDK in the enclosed list of references. Microsoft has also published sample code for intercepting traffic at this level.
C) External devices outside the operating systems. There are hardware products that are essentially small boxes attached to the back of the computer, or even embedded on network adapters, that see all network traffic going through them. Such devices have been used at least in security applications to implement functionality that could alternatively be done by intercepting traffic in the operating system.
The known solutions that fall into the above-mentioned categories A) to C) have not provided good, high-performance, robust solutions that would work on all widely used operating systems. In particular, many vendors have found it extremely difficult to develop packet interceptors for the Windows 95 and Windows 98 operating systems, which are currently very widely used and will remain so for several years to come.
Almost all software products that perform packet interception use intermediate drivers to perform the interception. FIG. 1 is a simplified block diagram that illustrates the known use of an intermediate driver especially in association with a Windows NT operating system. At the top of FIG. 1 there is an application program that has a user-mode client part 101. Between it and a network protocols block 106 there may be intermediaries which are of little significance to the present invention. The network protocols block 106 implements the network protocols, for example the TCP/IP protocol stack. The intermediate driver 107 resides between the protocol stacks and a NIC driver 108; it is separated from them through the NDIS interface the parts of which are separately shown in FIG. 1 as 102 and 103. The NIC driver block 108 is arranged to directly manage a NIC or Network Interface Card 109. The latter is a hardware component, usually an extension board coupled to the internal parallel bus of a computer. The NIC driver 108 may be referred to more generally as a network adapter. The NIC driver 108 allows upper layers to send and receive packets through the network and to perform control operations like handling interrupts, resetting or halting the interface card 109. It also allows the upper layers to query and set the operational characteristics of itself. The network adapter (i.e. the NIC driver 108) sees the intermediate driver 107 as if it was a protocol, and to the protocols block 106 the intermediate driver 107 behaves like a network adapter. The same NDIS interface is used on both sides of the intermediate driver, i.e. between blocks 106 and 107 on one hand and between blocks 107 and 108 on the other hand. The network interface card 109 is coupled to at least one physical transmission medium 110, which may be for example an optical fiber cable, a coaxial cable or a twisted pair of wires.
The intermediate driver approach works well for Windows NT and Solaris, which have been the principal platforms for Firewall and VPN applications. However, within the last year there has also been increased attention on implementing similar functionality in other operating systems, particularly Windows 95 and Windows 98. However, developing intermediate drivers for these systems has turned out to be extremely challenging.
A major problem in programming such drivers for these systems has been support for dial-up interfaces, as described for example in the reference marked as Simpson94 int eh enclosed list of references. A dial-up interface uses a serial port and modem to connect to the Internet or other network over the telephone network. Such interfaces are characterized by dynamic network addresses and the network connection being available only part of the time. The connection goes up (into an active, connected state) and down (into a passive, disconencted state) as the user connects and disconects the link.
The biggest problem for writing interceptors for dial-up interfaces has been that Windows 95, Windows 98 and Windows NT use a Microsoft proprietary extensions to the NDIS interface to provide the added functionality needed for dial-up functionality and dynamic addresses. Microsoft employees have recommended that packet interception should not be implemented for these platforms, because of the nature of the proprietary interface. Many people have been known to try to write intermediate drivers for these platforms without success or with limited success only after years of reverse engineering and development.
A few companies have managed to implement intermediate drivers even for these platforms. However, very recently, intermediate drivers have also been found to have reliability problems. The drivers have not worked with all network adapters or all versions of the operating systems, and as more vendors are bringing products using intermediate drivers to the marketplace, people are experiencing serious compatibility problems between intermediate drivers from different vendors installed in the same computer. Getting the different intermediate drivers to bind to each other in the correct order is a hard, possibly unsolvable problem. These issues may eventually cause intermediate drivers to be too unreliable for general use.
A further problem with intermediate drivers is performance. The intermediate drivers add a substantial amount of processing to the data path of a network packet. Significant amounts of data copying may also take place. Thus, packet interception through the use of intermediate drivers is becoming an important performance bottleneck for large-volume applications. The NDIS library implements also some locking strategy on behalf of the intermediate drivers, which is likely to add overhead and causes protocol-intermediate-NIC code paths to be effectively half-duplex. This is especially bad for routing applications.
It is an object of the present invention to provide a method and arrangement for packet interception that would be widely compatible with different network adapters, different operating systems, different operating system versions and various third party software. It is an additional object of the invention to provide such a method and arrangement with the additional advantage of being easily adopted to use even by relatively unskilled users.
The objects of the invention are achieved by using a method we call hooking in a sophisticated fashion to get access to all the necessary packets and information related to them, and by providing an arrangement that is programmed to implement such a method.
In its first embodiment the method according to the invention comprises the steps of
providing a set of replacement functions within a packet interceptor module;
hooking at least one function used for transmitting network packets from a first protocols entity to a first network adapter into a first replacement function;
hooking at least one function used for transmitting network packets from said first network adapter to said first protocols entity into a second replacement function; and
hooking at least one function used for receiving information about the status of the network interface implemented by said first network adapter into a third replacement function.
In a second embodiment the method according to the invention comprises the steps of
providing a set of replacement functions within a packet interceptor module;
hooking a plurality of functions used for transmitting network packets from protocols entities to network adapters into a first set of replacement functions;
hooking a plurality of functions used for transmitting network packets from network adapters to protocols entities into a second set of replacement functions; and
hooking a plurality of functions used for receiving information about the status of the network interfaces implemented by network adapters into a third set of replacement functions.
In a third embodiment the method according to the invention comprises the steps of
replacing a first operating system module with a certain first replacement module that implements a programming interface equal to a programming interface of the first operating system module and calls said first operating system module from a plurality of the entry points of the programming interface;
using said replacement module to identify at least one network adapter and at least one protocols entity installed in the computer system;
using said replacement module to replace at least one function used for transmitting network packets from said first protocols entity to said first network adapter;
using said replacement module to to replace at least one function used for transmitting network packets from said first network adapter to said first protocols entity;
using said replacement module to replace at least one function used for receiving information about the status of the network interface implemented by said first network adapter;
using said replacement module to determine, whether or not a dynamic IP address has been allocated for the network interface implemented by said first network adapter; and
in a case where a dynamic IP address has been allocated for the network interface implemented by said first network adapter, using said replacement module to determine, which said dynamic IP address is.
Additionally the invention concerns a computer system for handling network packets, comprising
a first network adapter arranged to implement a network interface;
a first protocols entity;
a number of predetermined functions for communicating network packets between said network adapter and said protocols entity;
a packet interceptor module for determining a set of replacement functions;
within said packet interceptor module, means for hooking at least one function used for transmitting network packets from said first protocols entity to said first network adapter into a first replacement function;
within said packet interceptor module, means for hooking at least one function used for transmitting network packets from said first network adapter to said first protocols entity into a second replacement function; and
within said packet interceptor module, means for hooking at least one function used for receiving information about the status of the network interface implemented by said first network adapter into a third replacement function.
In an even further embodiment the invention concerns a packet interceptor module for intercepting network packets in a computer system which comprises a first network adapter, a first protocols entity and a number of predetermined functions for communicating network packets between said network adapter and said protocols entity; said packet interceptor module comprising
the definition of a set of replacement functions;
means for hooking at least one function used for transmitting network packets from said first protocols entity to said first network adapter into a first replacement function;
means for hooking at least one function used for transmitting network packets from said first network adapter to said first protocols entity into a second replacement function; and
means for hooking at least one function used for receiving information about the status of the network interface implemented by said first network adapter into a third replacement function.